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The constructs used by scholars across the physical, natural, and social sciences to 
evaluate research quality-as objective, reliable, valid, generalizable, randomized, 
accurate, authentic-are not value-free or apolitical. They all require the application of 
human judgment, which is inevitably affected by cultural norms and values (Westmeyer, 
1981). Consequently, the use of these constructs to assess research focused on 
minority, marginalized populations requires extraordinary judgment. Such assessments 
must be based not only on Western notions of scientific quality but also, in the case of 
American Indians and Alaska Natives (AI/ANs), on a separate set of criteria prescribed 
in the interest of sovereignty. Sovereignty refers to the inherent, as well as 
constitutionally recognized, rights of tribes to self-government, self-determination, and 
self-education. Of course no contemporary polity-whether American Indian tribe or U.S. 
federal government-exercises unlimited sovereignty: "In the real world, sovereignty 
operates within constraints" (Wilkins & Lomawaima, 2001 , p. 5). This Digest addresses 
these issues as researchers take on the challenges set forth by the "American Indian 
and Alaska Native Education Research Agenda" (Research Agenda Working Group, 
Strang, & von Glatz, 2001 ). The perspectives in this Digest are those of the authors and 
do not necessarily represent the positions of the U.S. Department of Education. 

DEFINITION OF TERMS 



In the sciences, the terms "accuracy," "reliability," and "validity" are used to make 
evaluative judgments of measurement instruments (e.g., tests and surveys), or of data 
collection methods. An "accurate" measurement is one that is precise and exact. 
Although commonly applied to the mechanical measurements of the physical sciences, 
accuracy is also an issue in social science, including qualitative research, where 
researchers seek to understand complex, multiple perceptions of reality. A "reliable" 
method or instrument gives consistent results in different applications. Consistency, 
however, "says nothing about being right or wrong" (Kelly, 1996, p. 43). The "rightness" 
or "truth value" of research is an issue of "validity." Validity is always subject to human 
judgment: it asks the question, "Does this device [or method] measure what it is said [or 
claims] to measure?" (Westmeyer, 1981, p.42). 

The concept of validity has been contested and is subject to further refinement. In 
experimental research, it is common to distinguish "internal validity" (whether a 
proposed cause is actually the reason for an effect) from "external validity" (whether 
research results are generalizable across settings or populations) (Miller & Salkind, 
2002). However, some qualitative researchers reject validity as a criterion forjudging 
research, arguing that understanding is a more proper concern (Wolcott, 1994). Maxwell 
(1992) argues that validity in qualitative research refers not to data or methods but to 
accounts. "Descriptive validity" tests the factual accuracy of accounts; "interpretive 
validity" asks whether an account represents research participants' perspectives; 
"theoretical validity" refers to the explanatory power of accounts; and "evaluative 
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validity" questions the usefulness or applicability of accounts. Parallels can be found in 
Banks' (1997) assessment of special education needs among AI/AN children. She 
distinguishes "construct validity" (the meaningfulness of research constructs or ideas), 
"treatment validity" (whether research participants judge its objectives, instruments, and 
procedures acceptable), and "ecological validity" (the extent to which research looks 
beyond the individual to social-historical contexts). 

THE REQUIREMENTS OF SOVEREIGNTY AND 
SELF-DETERMINATION 



Judging research adequacy within the larger social-historical context of AI/AN education 
involves answering questions critical to the exercise of sovereignty: Why do the 
research? What factors motivate the researcher? Who has set the research goals? Who 
has the "disciplinary authority" (Page, 2000, p. 23) to do the research? Who will be 
involved in conducting the research? Whom does the research serve? How will it benefit 
the local community? 

Until recently, Native communities were rarely involved in raising these questions or in 
formulating, implementing, or evaluating studies. In the past few decades, however, 
tribal governments have increasingly taken control of research site access, instituting 
protocols and approval processes for research projects (Lomawaima, 2000; see 
Romero, 1994, for an excellent example of research working within protocols). For 
example, both the Tohono O'odham and Pascua Yaqui language policies stipulate that 
tribal authorities must approve all research about their people. The Yaqui policy lays out 
further requirements for copyright of publications growing out of such research (Zepeda, 
1990). Additional sovereignty safeguards include requirements that tribal 
representatives be involved in setting research standards and evaluating projects 
against those standards. All these measures have grown out of the recognition that 
local communities have as great a stake as outside scholars in establishing what 
constitutes high-quality, useful research. 

There are viable alternatives to conventional, colonizing research paradigms (Smith, 
1999). As a means for improving post-World War II intergroup relations, sociologist Kurt 
Lewin (1946) introduced an action research cycle of problem identification, fact finding, 
execution, and evaluation. A decade later, Corey (1953) and Taba and Noel (1957) 
operationalized this cycle in education. Now treated as synonymous with teacher 
research, action research is a "reemerging tradition" in which those previously 
designated as subjects actively participate in research processes that benefit them 
directly (Stringer, 1996, p. 7). Recent examples of teacher/action research in AI/AN 
education include Lipka, Mohatt, and the Ciulistet (teacher-leaders) Group's work on 
incorporating Yup'ik knowledge into science and math instruction (Lipka et al., 1998); 
collaborative research to revitalize Indigenous languages (McCarty, Watahomigie, & 
Yamamoto, 1999); and research on alternative literacy assessment and curriculum 
reform undertaken by Navajo teacher-researchers (McCarty & Dick, 2003; McCarty, 
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2002; for an overview of teacher research methods, see McCarty, 1997). 

AUTHENTICITY 



Are the quality and authenticity of research guaranteed by the involvement of tribal 
members in design and evaluation? Some say it takes even more. Swisher (1996) 
argues for the primacy of Native scholars in conducting research on Native peoples and 
issues, attributing to them the benefits of an insider's view and experiences, enhanced 
passion and commitment, and the "authority to ask new and different questions" (p. 93). 
She calls on non-Native researchers and mentors, whose expertise and high-quality 
work she readily acknowledges, to step aside in favor of Native authors. Many would 
agree with Swisher's call to increase the authority of, recognition of, and publication by 
Native scholars and researchers, but not all are willing to ask non-Native allies to step 
aside (see Mihesuah, 1996 for a range of opinions). These observers suggest that the 
most productive research will result from respectful collaboration that does not 
dichotomize researchers as Native or non-Native but does make the contributions of 
Native colleagues integral to the design and conduct of studies. 

Even those who adhere to the "Native-only" requirement run up against debates over 
who is "really Indian." Snipp (2000) productively discusses the many criteria used to 
identify and assign "Indian-ness" and tribal identity, and the political and other agendas 
attached to them. Snipp concludes that contemporary criteria for tribal 
membership-however historically situated and flawed-are the prerogatives of 
sovereign tribes. 

Local evaluations of authenticity, whether dependent on research subject or research 
investigator, should be respected in accord with the principle of self-determination, even 
as researchers struggle to work out their often anomalous positions as "insider," 
"outsider," or some combination of both (Brayboy & Deyhle, 2000). 

POLITICS AND SCHOLARSHIP 



There have been many calls over the years for improved research in AI/AN education. 
The most recent is Executive Order 13096 (Clinton, 1998), which mandates the 
development and implementation of a national research agenda in Indian education. 
Consultations and meetings with Native peoples across the country culminated in the 
"American Indian and Alaska Native Education Research Agenda" (Research Agenda 
Working Group, Strang, & von Glatz, 2001). The agenda identifies high-priority research 
interests and outcomes for Native communities and clearly articulates the central roles 
that Native scholars, parents, teachers, schools, and communities must play to ensure 
high-quality research. The promise embedded in the research agenda, however, may 
be threatened if limited notions of what constitutes high-quality research, using criteria 
such as those outlined earlier, overtake the powers of Native peoples to evaluate 
research in their communities and schools. 

Science as practiced by many Western researchers is often invoked as the answer to 
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educational woes. This view tends to dismiss the power of all human inquiry-including 
Indigenous knowledge systems-to observe, analyze, theorize, and generate creative 
solutions. Research and development projects designed within the Western 
mold-especially those meant to provide guidance for classroom practice-often rely on 
standardized scripts to instruct and standardized tests to assess effectiveness (Metcalf, 
2002). Using standardized tests with minority populations is a dubious practice (Padilla 
& Lindholm, 1995, p. 97). 

A much more promising line of research draws upon funds of knowledge in Native 
communities to derive and apply research insights in education. "Funds of knowledge" 
refers to the repositories of knowledge residing in communities that are frequently 
overlooked or dismissed in conventional Western science and school curricula (Moll et 
al., 1 992). In a study of the Yupiit (Alaska) Nation and school system, Yupiaq 
researcher Oscar Kawagley shows how local scientific, mathematical, and technological 
knowledge can be used to improve the school curriculum. Situating science within a 
Yupiaq worldview and ecology of sustainability, Kawagley provides a rigorous and 
authentic framework for research and "for rethinking what we teach in schools and how 
we teach it, particularly as it relates to science" (1995, p.138). 

Research is needed that attends to the great diversity among Nations and the variability 
within local sites. Such research will most often be qualitative in its methodology. Page 
(2000) points out that "Qualitative research has challenged the 'science' in the social 
sciences, and in the natural sciences, too, even though science has long been the gold 
standard for knowledge and a source of disciplinary authority" (p. 23). She warns 
against "nostalgia for the large-scale, randomized experiments thought to provide 
unbiased and exact answers" (p. 24). The quest for scientific truth in such endeavors 
will likely prove elusive (Cassell, 2002) and could pose a threat to AI/AN educational 
self-determination and sovereignty. 

The political context of educational research can make the difference between a project 
that serves Native community and strengthens sovereignty or actively undermines both. 
None of the parties involved in educational research is apolitical-not tribal communities, 
tribal governments, federal education agencies, or academicians. We all work from 
particular positions, but certain positions cannot continue to be privileged over others. 
Ultimately, judgments about research quality in Indian education most productively 
belong in Indian country. 
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